Vision-based grasping of objects from a table using the humanoid robot Nao

Introduction: There is a large amount of work in robotics that addresses hand-eye coordination ([1], [2]) and vision-based calculation of objects and of grasping pose, but autonomous grasping on Nao poses particular problems: We face rather limited on-board processing power, inherent inaccuracies in repositioning due to the building quality of the robot, reduced stiffness of joints in case of continuous operation, a quite limited grasping ability of Nao’s hands with their pincer-shaped three fingers, as well as severe self-occlusion when the hand approaches the object in its final grasping position (the robot’s camera will mostly see the back of the hand with self-occlusion of the fingers and severe occlusion of the object). Therefore, we propose a solution to autonomous vision-based grasping of objects from a table that consists of: